<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN"><html><head>
	<meta http-equiv="CONTENT-TYPE" content="text/html; charset=iso-8859-1"><title>MVAPICH Integration</title>
	
	<meta name="GENERATOR" content="StarOffice 6.0  (Solaris Sparc)">
	<meta name="AUTHOR" content="Christopher.Duncan@Sun.COM">
	<meta name="CREATED" content="20030729;9041800">
	<meta name="CHANGED" content="20030729;9165000"></head>

<body lang="en-US">
<h1 style="text-align: center;">MVAPICH</h1>
<h1 style="text-align: center;">Grid Engine Parallel Support for MVAPICH</h1>
<h1 style="text-align: center;">with Tight and Loose Integration</h1>
<div style="text-align: center;"><br>
</div>
<br>
<br>
<br>
<br>
<h2>Content</h2>
<ol>
  <li>Content of mvapich-1.0.tar.gz</li>
  <li>mvapich.template</li>
  <li>mvapichl.template</li>
  <li>startmpi.sh</li>
  <li>stopmpi.sh</li>
  <li>'mvapich.sh' job example</li>
  <li>'mvapichl.sh' job example</li>
  <li>Copyright<br>
  </li>
</ol>
<h2>1. Content of mvapich-1.0.tar.gz</h2>
The <a href="/files/documents/7/30/mvapichsge-1.0.tar.gz">mvapichsge-1.0.tar.gz</a>&nbsp; archive contains the following files and directories:<br>
<table cellpadding="2" cellspacing="2" border="1" style="text-align: left; width: 100%;">
  <tbody>
    <tr>
      <td style="vertical-align: top;">README</td>
      <td style="vertical-align: top;">this file</td>
    </tr>
    <tr>
      <td style="vertical-align: top;">startmpi.sh</td>
      <td style="vertical-align: top;">startup script for MVAPICH<br>
      </td>
    </tr>
    <tr>
      <td style="vertical-align: top;">stopmpi.sh</td>
      <td style="vertical-align: top;">shutdown script for MVAPICH</td>
    </tr>
    <tr>
      <td style="vertical-align: top;">mvapich.template</td>
      <td style="vertical-align: top;">a MVAPICH PE template configuration for Grid Engine (tight)</td>
    </tr>
    <tr>
      <td style="vertical-align: top;">mvapichl.template</td>
      <td style="vertical-align: top;">a MVAPICH PE template configuration for Grid Engine (loose)</td>
    </tr>
    <tr>
      <td style="vertical-align: top;">mvapich.sh</td>
      <td style="vertical-align: top;">a sample MVAPICH job&nbsp; (tight)</td>
    </tr>
    <tr>
      <td style="vertical-align: top;">mvapichl.sh</td>
      <td style="vertical-align: top;">a sample MVAPICH job&nbsp; (loose)</td>
    </tr>
    <tr>
      <td style="vertical-align: top;">hostname</td>
      <td style="vertical-align: top;">a wrapper for the hostname command</td>
    </tr>
  </tbody>
</table>
<br>
 Please refer to the "Administration and User Guide" Chapter "Support
of Parallel Environments" for a general introduction to the Parallel
Environment Interface of Grid Engine. (<a href="http://docs.sun.com/source/816-4739-11/chp10.htm">Grid Engine 5.3</a>, <a href="http://docs.sun.com/app/docs/doc/817-5677/6ml49n2bu?q=grid+engine&amp;a=view">Grid Engine 6.0</a>)<br>
<br>
<br>
<h2>2. mvapich.template<br>
</h2>
Use this template as a starting point when establishing a parallel
environment for MVAPICH with tight integration. You need to replace
&lt;a_list_of_parallel_queues&gt;, &lt;the_number_of_slots&gt; and
&lt;your_sge_root&gt;&nbsp; with the appropriate information.<br>
<br>
Here is a list of problems for which tight integration provides solutions<br>
<ul>
  <li>resource limits are enforced also for tasks at slave hosts</li>
  <li>resource consumption at slave hosts can be accounted</li>
  <li>no need to write a customized terminate method to ensure that whole job is finished on qdel<br>
  </li>
</ul>
Here is a list of problems which are not solved by the tight integration<br>
<ul>
  <li>can't trigger job finish if application finishes partially</li>
</ul>
<h2>3. mvapichl.template</h2>
Use this template as a starting point when establishing a parallel
environment for MVAPICH with loose integration. Not recommended since
MVAPICH doesn't clean its jobs properly in this mode when, for example,
qdel is used.<br>
<br>
<h2>4. startmpi.sh</h2>
The starter script 'startmpi.sh' needs some command line arguments, to
be configured by use of either qmon or qconf. The first one is the path
to the "$pe_hostfile" that gets transformed by startmpi.sh into a MPI
machine file. On successful completion startmpi.sh creates a machine
file in $TMPDIR/machines to be passed to "mpirun" at job start.<br>
<br>
$TMPDIR is a temporary directory created and removed by the Grid Engine execution daemon.<br>
<br>
<h2>5. stopmpi.sh</h2>
The stopper 'stopmpi.sh' just removes $TMPDIR/machines. &nbsp;<br>
<br>
<h2>6. 'mvapich.sh' job example</h2>
The job example 'mvapich.sh' starts the xhpl' program. Please note that
a MPI job that has to start 'mpirun_rsh' with the options&nbsp; "-np
$NSLOTS" to start the job with the correct number of slots ($NSLOTS is
set by Grid Engine). <br>
To pass information where to start the MPI tasks one has to pass&nbsp;
"-hostfile $TMPDIR/machines" as the second argument. &nbsp;<br>
<br>
Additionally, for tight integration remember to use "-rsh " and
optionally, you can use "-nowd" to prevent mvapich to 'cd $wd' in the
remote hots. <br>
This leaves SGE in charge of the working directory.<br>
<br>
<h2>7. 'mvapichl.sh' job example</h2>
Is the same case, but using loose integration, where the option "-ssh" can be used instead of the "-rsh".<br>
<br>
<h2>8. Copyright</h2>
Marcelo Matus<br>
<br>
</body></html>
